Papers with Instruction-tuned language models
Data Selection for Multi-turn Dialogue Instruction Tuning (2026.findings-acl)
Copied to clipboard
| Challenge: | Instruction-tuned language models often use noisy multi-turn dialogue datasets with topic drift, repetitive chitchat, and mismatched answer formats across turns. |
| Approach: | They propose a dialogue-level framework that scores whole conversations rather than isolated turns. |
| Outcome: | The proposed framework outperforms strong single-turn selectors, dialogue-level LLM scorers and heuristic baselines on three multi-turn benchmarks and an in-domain Banking test set. |
Diversification Catalyzes Language Models’ Instruction Generalization To Unseen Semantics (2025.findings-acl)
Copied to clipboard
| Challenge: | Instruction-tuned language models excel in knowledge, reasoning, and instruction-following . however, the factors enabling generalization to unseen instructions remain underexplored . |
| Approach: | They propose to model instruction-following as a computational process and design controlled experiments inspired by the Turing-complete Markov algorithm to disentangle its dynamics. |
| Outcome: | The proposed model outperforms scaling up data volumes in generalist models by combining in-domain and diverse out-of-domain tasks. |